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We consider the problem of conducting a survey with the goal of obtaining an unbiased esti- 
mator of some population statistic when individuals have unknown costs (drawn from a known 
prior) for participating in the survey. Individuals must be compensated for their participation 
and are strategic agents, and so the payment scheme must incentivize truthful behavior. We 
derive optimal truthful mechanisms for this problem for the two goals of minimizing the variance 
of the estimator given a fixed budget, and minimizing the expected cost of the survey given a 
fixed variance goal. 

q 

1 Introduction 

Consider the idealized process of conducting a survey: individuals x can be randomly sampled 
from some population X, and can be asked a question q for some q : X — > {0, 1}. If the goal 
is to estimate E xe x[?(^)]) then the empirical average of q(x) gives us an unbiased estimator, and 
indeed, only 0(l/a 2 ) samples are required to estimate this quantity to within accuracy ±a with 
^D ■ high probability. 

However, as with many things in life, conditions are rarely ideal. Although it may be possible 
to randomly sample individuals from a population, it is not always possible to convince them 
to participate in a survey. Participation is not without cost, and individuals are self interested. 
Therefore, although the population sampled for the survey may be random, the population that 
actually responds to the survey is biased towards the subset of the population that has a low cost for 
participating, which may significantly distort the results. This problem can be partially alleviated 
by compensating individuals for their participation. But what is the right level of compensation? 
For a variety of reasons, including varying time constraints and privacy concerns, the cost for 
participating in a survey can vary substantially from individual to individual. Any fixed level of 
compensation.!] will therefore still result in a biased sample. We assume that either the agents will 
not lie about the content of the survey, or that their responses are somehow otherwise verifiable. 

On the other hand, costs for participation are known only to the individuals and not to the 
surveyor, and so it is not possible to simply pay each sampled individual exactly their cost for 
participation. They cannot simply be asked their cost, because as rational agents, they can mis- 
report their true costs if it is financially beneficial for them to do so. The surveyors problem 
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therefore lies in the realm of mechanism design: he must determine a payment rule that properly 
incentivizes individuals from the population to report their true cost for participation. 

In this paper, we initiate the formal study of the surveyors problem from the perspective of 
mechanism design. We consider the problem of computing an unbiased estimator of some statistic 
^xex [q( x )} when individuals from X have varying costs for participation drawn from a known prior 
J-. We derive optimal truthful mechanisms for minimizing the variance of the estimate given a fixed 
budget on the expected cost of the survey, and for minimizing the expected cost of the survey given 
a fixed constraint on the variance of the estimator. Our mechanism is simple, practical, and ex-post 
dominant strategy truthful. 

1.1 Our Results 

Given a prior distribution on costs J-, we show how to conduct an survey that optimally trades 
off the expected variance^ of an unbiased estimator with the expected cost. We first prove a 
characterization theorem that greatly simplifies our design space: we show that for any truthful 
surveying mechanism, there exists another truthful surveying mechanism that merely makes take-it- 
or-leave-it offers that constructs an identical estimator, and has the same expected cost. Therefore, 
we may without loss of generality restrict our attention to take-it-or-leave-it survey mechanisms 
when seeking to optimize any function of the survey's estimator as it relates to the expected cost 
of the survey. 

Then, under an extremely mild regularity condition on the prior distribution on costs J 7 , we 
derive the form of a surveying mechanism which optimizes any continuous, convex, and monotone 
function of the cost and variance of the mechanism. In particular, this allows us to derive the 
surveying mechanism which achieves the minimum variance unbiased estimator of a population 
statistic subject to a cost constraint, and similarly, the minimum cost mechanism subject to a 
variance constraint. 

Because we are optimizing over the class of take-it-or-leave-it mechanisms (to be defined), our 
optimization problem reduces to finding the optimal distribution over offer prices. We note that 
if this were an unconstrained optimization problem, it would end up being a relatively simple 
exercise in the calculus of variations. However, because we must solve for a function which is a 
valid probability distribution, we are facing a constrained optimization problem which results in a 
number of technical complications. 

1.2 Related Work 

The problem of survey design has a long history, and is the topic of an entire field of inquiry in 
statistics. The idea that survey results may be biased because different individuals may have differ- 
ent proclivities for answering the survey as a function of their types (due to stigmatizing questions) 
goes back at least to Warner [War65| . who proposed a "randomized response" approach to lessen 
the cost of answering a stigmatizing question. The statistical literature tends to approach the 
problem by making distributional assumptions about the relationship between survey response and 
data type, and attempts to avoid bias by compensating for these differences or imputing the values 
of missing data. We do not attempt to survey this vast literature, but refer the interested reader 
to useful texts on this topic |LL91llSMOGll] . In particular, our work differs from this literature in 
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that we do not make any assumptions about the relationship between data value and response rate, 
but instead take a mechanism-design approach and model survey respondents as rational agents 
with private costs for participating in the survey, and insist on truthful survey mechanisms which 
incentivize sufficient response rate to provide accurate statistical estimates, without assumptions on 
how costs are related to types. Crucially, we are prepared to pay individuals for their participation, 
and rely on their rational responses to guarantee the accuracy of our estimators. 

The field of Bayesian optimal mechanism design originated with the groundbreaking work of 
Myerson |Myc81|, which (in certain settings) characterized the auction rule which maximizes seller 
revenue, given a known prior on the distribution over buyer values. See, e.g., [Kri 09] for a survey 
of the literature that has followed. In this work, we take the methodology of Bayesian optimal 
mechanism design and apply it to the problem of survey design. We assume a known prior over costs 
for individuals participation, but assume that the correlation between costs and survey responses is 
worst-case. We then design truthful survey mechanisms for calculating unbiased estimators while 
minimizing various objective functions (i.e. minimizing the variance of the estimator given a cost 
constraint, or minimizing the cost of the survey given a variance constraint, as well as more general 
objectives). 

Recently, the problem of designing truthful mechanisms to estimate statistics from a population 
which explicitly experiences costs for privacy loss was introduced by Ghosh and Roth |GR11| . 
Subsequently (and concurrently with this work), Ligett and Roth [LR12] extend this work to 
sequences of take-it-or-leave-it offers. Although it has similar goals, this line of work differs from 
the current paper in that it measures cost using the formalism of differential privacy, and more 
importantly, takes a worst-case view and does not assume a known prior over agent costs. In 
contrast, here we use a prior over agent costs to derive optimal mechanisms. This paper can be 
viewed as answering an open question posed by [GR11] . which asked whether the approach of 
Bayesian-optimal mechanism design could be brought to bear on the data gathering problem when 
the distribution over agent costs was known. 

2 Preliminaries 

2.1 Model and Mechanism Design Basics 

We consider a surveyor who is interested in some population statistic defined by a predicate on 
individuals, q : X — > {0, 1}. We model each individual X{ £ X as having an unknown cost Cj > for 
participating in the survey (i.e. for revealing the bit q(xi)). Individuals and costs (xj,Cj) are drawn 
i.i.d. from a distribution V: (xj,Cj) ~ V, and the surveyor wishes to estimate 'Ki Xifii \^,- D [q{xi)\. 
Note that q and Xi can be arbitrarily correlated. We denote by F the marginal distribution of T> 
on c. We write / for the probability density function for F, and write F for the cumulative density 
function. We assume that / is piecewise differentiable except, possible, on a measure set. The 
surveying mechanism knows (and can be a function of) F, the marginal distribution on costs, but 
has no other knowledge of T>. We write n to denote the number of individuals sampled by the 
surveying mechanism. 

A survey mechanism consists of an allocation rule A : R + — > [0, 1] and a possibly randomized 
payment rule P : R + -> R> . We define P : R + -> M> to be P(c) = E[P(c)]. At time i, 
the mechanism can sample an individual (xj,Cj) ~ T>, elicit their reported cost Cj, and then with 
probability A{ci) collect the survey sample q{xj) in exchange for a payment P(di). With the 



remaining probability 1 — A(di) the mechanism does not conduct the survey and does not make 
any payment. 

We model individuals as having quasi-linear utility functions and being expectation maximizers. 
An individual with cost q who reports cost t\ to a surveying mechanism, experiences cost -P(q) — q 
when the survey is conducted, which occurs with probability A(cj). Therefore their expected utility 
is: 

u(a, A(di), P(di)) = A(di) ■ (P(di) - a) 

Note that this only depends on P via its expectation P. We want our surveying mechanisms to 
be able to give accuracy guarantees even in the presence of rational, selfish agents who wish to 
maximize their own utility. Therefore we require that our mechanisms be truthful and individually 
rational. 

Definition 2.1 (Truthfulness). A surveying mechanism (A,P) is truthful if for all Ci,di £ R + : 

u(ci,A(a),P(ci)) > u(ci,A(ci),P(ci)) 

Definition 2.2 (Individual Rationality). A surveying mechanism (A,P) is individually rational if 
for all Cj resulting in the survey being conducted: 

P(Ci) > h 

Informally, truthfulness states that a utility-maximizing agent can do no better than report- 
ing his true cost faithfully to the surveying mechanism, and individual rationality states that no 
individual can ever be forced to experience negative utility. 

Truthfulness and individual rationality impose the constraints on our design space for truthful 
surveys. We now define the objective that we wish to maximize within this space. When a 
transaction is made with an individual % through the mechanism (which occurs with probability 
A(ci)), he supplies his bit q(xi) £ {0, 1}. The mechanism compiles these bits into an estimator S 
of the statistic ^< x A~v[<l{%)\- We require that our mechanisms give unbiased estimates. 

Definition 2.3 (unbiased estimator). An estimator S = S(T>,A,n) of the statistic E( z c )~v[l(, x )] 
is unbiased if: 

E[S(V,A,n)]=E {x ^ v [q(x)} 

where the randomness of E[S(T>, A, n)] is taken over the random choices of the sampled individuals 
(xj,Cj) ~ T> for i £ [n] and the internal randomness of the mechanism allocation rule A. 

We wish to design mechanisms for constructing unbiased estimators of E^^^^x)] while 
minimizing variance. 

Definition 2.4 (variance). The variance of a random variable S(T>,A,n) is: 

V&r(S(V, A, n)) = E[(S(T>, A, n) - E[S(V, A, n)]) 2 ] 

Note that Var(S') depends on the unknown distribution £n, even though the mechanism knows 
only the marginal distribution over costs, T . Therefore, our measure of performance will be worst 
case variance over all distributions T> consistent with the marginal distribution T\ 



For example, most reasonable estimators S(T>, A, n) will have Var(S) = if T> is such that Pr ( x ,c)~t> [<l(z) = 1] = 0. 



Definition 2.5 (Worst case variance). The worst case variance of a random variable S(T>,A,n) 
given a marginal distribution on costs J- is: 

Var*(5) = maxVar(S'(P, A,n)) 

where the maximum is taken over all distributions T> consistent with T . 

When the distribution is not known, it is not always possible to design the minimum variance 
unbiased estimator of a distribution parameter. Therefore, we will concentrate on linear estimators 
in this paper. 

Definition 2.6. An estimator S is linear if for each individual i from which the mechanism elicits 
q(xi), there is a multiplier /3(cj) such that: 

S = Y^ P{ci)q(xi) 

We will want to measure the expected cost of running a truthful survey. The mechanism 
only needs to make a payment P(cj) when a transaction with agent i occurs, which happens with 
probability A(ci). Therefore we can derive the expected cost of a truthful mechanism with prior 
T . The following claim is immediate: 

Claim 2.1. The expected cost of surveying n individuals with a truthful mechanism (A,P) is: 

POO 

E[cost(A, P,n)]=n- / A(c) ■ P{c) ■ /(c) dc 
Jo 

For intuition's sake, we here give an example of a non-trivial truthful lottery mechanism: 

Example 2.1. For values Cj £ [1, oo], consider a lottery mechanism with A(x) = 1/x 2 and P(x) = 
2x. Note that we always have P(x) > x and so the mechanism is individually rational. Observe 
that it is also truthful. For any individual i, we have: 

1 2 c- 

Ui(Ci,A(Ci),P(Ci)) = ^2 (2cj -Ci) = — --Z2 

taking the derivative with respect to Cj, we can see that this quantity is always maximized when 
Cj = Cj, which is the condition needed for truthfulness. 

2.2 Calculus of Variations Basics 

In this paper we will seek to minimize the cost and variance of our mechanisms, which are functionals 
of our chosen allocation rule A and pricing rule P. Functional minimization problems are addressed 
by the Calculus of Variations. Here we give an informal introduction to the required (very basic) 
preliminaries. This can be found, for example, in [Lib 12]. 

Let U be a space of functions over which we would like to optimize. Let M : U — > R. The first 
variation (which is corresponds to the Gateaux derivative) of M at G E U in the direction G is 
simply 



8M\ G (G) = lim 

1 e— >0 



M{(G + eG)-M{G) 



A direction is feasible if G + eG € U for sufficiently small e. We only deal with convex U. In 
this case every feasible direction of a derivative at G can be written as G — G for some G £ U. 

For a convex [/, we say that G is a minimizer (or local minimum) of M if for every feasible 
direction G we have that 8M\q[G) > 0. 

For a convex U, we call 5 2 Miq : U — > R the second variation of M at G if 

M(G + eG) = M(G) + e<5M| G (G) + e 2 <5 2 M| G (G) + o(e 2 ) 

for all feasible directions G £ C7 and all e > 

M is convex if <5 2 M| G > for all G € [/. M is strictly convex if <5 2 M| G > for all G €U. If M 
and C/ are convex than any minimizer is also a global minimum. If M is strictly convex and U is 
convex than any minimizer is also a unique global minimum. 

We will use Lagrange Multipliers. If we are trying to minimize M : U x R — > R subject to 
the constraint ff:(/xl4l, then by properties of Lagrange Multipliers we must have that any 
direction that decreases M is in the tangent space of the constraints. Thus it is sufficient to show 
d G [M{G* , n) + XH (G* , n)] > 0, d x [M{G* , n) + XH{G* , n)} = 0, and d n [M(G* , n) + \H(G* , n)] = 
for some A6l. 

3 Simplifying the Design Space 

In this section, we show that without loss of generality, we can restrict our attention to a simple 
class of mechanisms. 

Definition 3.1. A take-it- or-leave-it (TIOLI) lottery mechanism is defined by a distribution Q 
over R + with probability density function g and cumulative density function G. Given a price x 
from an agent, A(x) and P(x) are then computed as follows: 

1. Sample pi ~ Q. 

2. Set A(x) = if x > pi and A(x) = 1 if x < Pi- 

3. Set P{x) = if x > pi and P(x) = pi if x < pi. 

Thus A and P are completely determined by Q. Namely A(c) = 1 — G(c); and P(c) is the 
distribution Q conditioned on being greater than c. 

We first observe that a TIOLI lottery mechanism is truthful and individually rational. 

Theorem 3.1. Any take-it- or-leave-it lottery mechanism is truthful and individually rational. 

Proof. It is immediate that TIOLI mechanisms are individually rational, since whenever A{ci) = 1 
we have q < pi and so: P((h) = Pi > Q- To see that the mechanism is truthful, observe that for each 
fixed pi , if Ci < pi then individual i gets utility pi — Ci for every reported valuation q > pi and utility 
for every reported valuation c\ < Pi. If q < pi then Pi — Ci > and the so the utility of reporting 
di = Ci is non-negative and is not changed by over-reporting c\ > Ci (and under-reporting can only 
reduce the payoff to 0). If q > pi then pi — Ci < 0, which cannot be improved by under-reporting 
Ci < Ci (and over-reporting can only result in negative utility). □ 

We now show that without loss of generality, we may restrict our attention to TIOLI lottery 
mechanisms: that is, every lottery mechanism (^4, P) is implementable as a TIOLI lottery mecha- 
nism with the same allocation rule and the same expected payment. 

6 



Lemma 3.2. If (A, P) is a truthful lottery mechanism, then A(x) must be monotonically decreasing 
in x. 

Proof. Let x\,x 2 6 K + . We will show that if A(x 2 ) > A(x\) then x\ > x 2 . By the truthfulness of 
the mechanism (A, P) we have that for all c: 

Ui (c, A(x),P(x)) = A{x){P{x) - c) = A(x)P(x) - A(x)c 

is maximized at x = c. In particular, considering c = x\ and c = x 2 we have the following pair of 
inequalities: 

A(a;i)P(xi) - A{x x )x x > A(x 2 )P{x 2 ) - A(x 2 )x x 

A(x 2 )P(x 2 ) - A(x 2 )x 2 > A(xi)P(xi) - A(xi)x 2 

Adding these two inequalities, we find: 

xi(A{x 2 ) - A(xi)) + x 2 {A(x 1 ) - A(x 2 )) > 

Note that by assumption, (A(x 2 ) — A(xi)) > 0. Therefore, dividing both sides by (A(x 2 ) — A(xi)) 
we have: 

xi — x 2 > 

which completes the proof. D 

Theorem 3.3. Every truthful differ entiable allocation rule A : M + —> [0, 1] is implementable as 
a take-it- or-leave-it lottery mechanism with the same allocation rule A{x) and the same expected 
payment ¥\A(x)P(x)\. 

Proof. Fix an allocation rule A* and a corresponding payment rule P* so that (A* ,P*) is a truthful 
lottery mechanism. Define a TIOLI lottery mechanism with allocation and payment rules denoted 
A, P respectively, by defining the distribution over offers Q to be the distribution with probability 
density function g(x) = — 4-A*(x). Note that by lemma 13.21 we have g(x) > for all x, and 
Jo s( x ) = ^4*(0) < 1, and so g is a valid probability density function. Now consider the probability 
that this TIOLI mechanism transacts with an individual i with cost q: this occurs exactly when 
the sampled offer pi is such that pi > Ci, which occurs with probability: 

poo poo J 

A(*) = / g(x)dx =- -r-A*(x) = A* (a) 

J Ci J Ci 

Thus the given TIOLI lottery mechanism implements exactly the allocation rule A. It remains to 
show that the expected payment of both mechanisms are the same: ELA(x)P(a;)] = A*{x)P*{x) for 
all x. 

E[A(x)P(x)] = E[P(x)\pi > x] ■ Pr[ Pi > x] 

t ■ g(t)dt 

Now consider the utility of agent i reporting value x to mechanism (A*,P*): 

u(a,A*(x),P*(x)) = A*(x) (P*(x) - a) = A*(x)P*(x) - CiA*{x)) 

7 



By the truthfulness of this mechanism, for every value of Cj, this expression is maximized by setting 
x = Cj. Write the function A*P*(x) = A*(x)P*(x). Taking the derivative of the above expression 
and setting it to zero, we have simultaneously for all values c: 

±-<A-P-(c )) = c.±<A-( c)) 
Integrating both sides of the above equality from x to oo we have: 

-A*{x)P*{x) = / -^(A*P*(c))dc 
Jx d c 

c ■ g(c)dc 



In other words, for all x: 

A*(x)P*(x) = I c • g(c)dc = E[A(x)P(x)} 



D 

Remark 3.1. The requirement that the allocation rule is differentiable is for technical convenience, 
and can be relaxed to a more mild piecewise differentiable condition. 

We have just seen that with respect to allocation and payment rules, we can restrict our attention 
without loss of generality to TIOLI mechanisms. What about unbiased estimators? In our setting, 
as it happens, the celebrated Horvitz-Thompson estimator is the unique unbiased linear estimator, 
and therefore the linear unbiased estimator of minimum variance, independent of the distribution 
V. 

Definition 3.2 (Horvitz-Thompson Estimator). For an allocation rule A(x), the Horvitz-Thompson 
estimator computes: 

1 n 

S HT (P,A,n) = -y2 d i 
n ^— ' 

8=1 

where: 

1 *? \ i If the mechanism transacts with individual^; 
I 0, Otherwise. 

Theorem 3.4 ( |HT52j ). For every distribution T> and every allocation rule A, SHT(T>,A,n) is the 
minimum variance linear unbiased estimator ofK[q(xi)]. 

Moreover, the Horvitz-Thompson estimator is admissible among the set of all unbiased esti- 
mators of ^( x c )~T>[l( x )) [GJ65] (even non-linear estimators). This means that the estimator is 
un- dominated: there is no other estimator that has lower variance than Sht with respect to every 
distribution D. 

Therefore, for the rest of the paper, we restrict our attention to TIOLI mechanisms that use 
the Horvitz-Thompson estimator. 



3.1 Price and Variance of TIOLI mechanisms that use the Horvitz-Thompson 
estimator 

In this section we derive the cost and variance of the TIOLI mechanisms that use the Horvitz- 
Thompson estimator. 

Recall that for a TIOLI mechanism with price distribution Q, we have: 

POO 

E[Cosb(A(g),P(g),n)] = n- / A(c) ■ P(c) ■ f(c)dc 

Jo 

poo / roo \ 

= n ■ /(c) • I / x ■ g(x)dx I dc 

roo 

= n ■ c ■ g{c) ■ F(c)dc 

Jo 

We can compute (up to additive error 1/n) the worst case variance of a TIOLI mechanism 
characterized by distribution g-. 

Lemma 3.5. The worst-case variance of a Horvitz-Thompson TIOLI mechanism parameterized 
with distribution g used on a marginal value distribution J- for n iterations is: 



1 f°° f(r) 1 1 f°° 

~ T 2 T7 n dx--<Var*(A(g),^,n)<- - 

n J 1 1 — G(x) n nJi 1 



/« * 



G(x) 

Proof. Recall that given an allocation rule A, the Horvitz-Thompson estimator is defined as: 



n 



Sht(D, A, n) = — y di where d 

<n £- — ^ 



qjxj) 



1 V^ j t, a J a i a » If the mechanism transacts with individual*; 



A( Ci ) 



n r~i 0, Otherwise. 



We will analyze Var(di) and then observe that Var(- Y17=i di) = -Var(di) since each di is 
independent and identically distributed. First note that Var(di) = E[df] — (E[di\) 2 . We observe 
that < E[di} 2 < 1 and that: 



r(* 



E[df] = / [f{v)-A(v)-Pi[q(x) = l\c = v]-[j^] ) dv 



1 ^ 



< C{m-^)-(-^)y v 

o A(v) 

oo 



'<"> * 



o 1-OW 

with equality holding for the distribution that sets q(x) = 1 independent of the valuation c. Hence, 
we have: 

p^rdu - 1 < Var (di) < / y^-rdv 

, 1-G(v) J l-G(v) 

as claimed. □ 



Let V*(A(Q), J 7 , n) = L 00 i"[q L\ dv. Instead of optimizing for the worst-case expected variance 
Vax* (A(Q) , J 7 , n) we will optimize for V*(A(Q), J 7 , n), since V* (A(G) , J 7 , n) is a very good approxi- 
mation and will be much more convenient. Additionally, we note that if we change the estimator 
slightly to be 

x /rr . . n 1 v^ r i i J ^n — n — , If the mechanism transacts with individuals; 

SHTiV.A.n) = - > di where di = { A (a) ' 

nf^ { 0, Otherwise. 

Then this is an unbiased estimator of 1— 2E[g(xj)] with expected variance exactly V* (A(G),F, n). 

For a TIOLI mechanism Cost(A(G),P(G),n) and V*(A(G), J 7 , n) can we written in terms of J 7 , 
Q and n, and no longer need to be stated in terms of T>, A, or P. Thus we will, in the future, 
denote them by Cost(F, G, n) and V* (J 7 , G, n), and will often put fixed terms in the subscripts (e.g. 

Costjr n (£)). 

Claim 3.6. Let J 7 , Q , and Q have piecewise twice differentiable CDFs, then 

5Cost Fn ^_ g {G-G)= / (g(y)-g(y))yF(y)dy 

Jo 

Proof. Calculations show 

Cost(F, (1 - e)G + eg, n) - Cost(F, Q, n 



lim 

e->0 



oo 

n J (g(y) - g{y))[yF(y)]dy 
o 



□ 



Claim 3.7. Let J 7 , G, and G have piecewise twice differentiable CDFs, the 



5V} Mg (G)= r(g(y)-g{y)) [* -, ,J^\ A , 2 dydx 
1 Jo Jo U y g(z)dz) 2 



Proof. Calculations show 



lim 

e^O 



Var(F, (1 - e)G + e$, n) - Var(F, G, n) 



1 f°° f(x)f™g(y)-g(y)dy 



2 



d,x 



nj (/~ g(z)dz) 

1 f°° f y fix) 

= -/ (g(y)-g(y)) 7700 , . , , 2 dxdy 

n Jo Jo (Jx 9{z)dzY 

This last step we switch the integrals, which we can do because / and g are piecewise continuous. 
The claim follows. □ 

4 Deriving Optimal Mechanisms 

In this section, we derive optimal TIOLI mechanisms that use the Horvitz-Thompson estimator. 
Recall that such mechanisms are defined by a distribution G over prices p G [0, oo) with CDF G 
and pdf g. In the previous section, we derived formulas for the cost, Cost(F,G,n), and worst-case 
expected variance, V*(F,G,n) with error at most ±l/n. 

10 



We now derive optimal mechanisms for minimizing the variance subject to a cost constraint, 
and minimizing cost subject to a constraint on the variance, where J 7 is a distribution that satisfies 
a certain regularity condition. In the case where the number of participants is fixed, we can extend 
this to design optimal mechanisms for any possible convex minimization objective function. 

We will assume that F is twice differentiable for all positive x, that n is fixed We remark that 
this assumption can be further relaxed so that / is piecewise twice differentiable. 

We optimize over the class of distributions with piece-wise twice differentiable CDF. Also, 
without loss of generality, we require that Q has no support less than every point in the support 
of J- except possibly a point mass at zero. This is clearly without loss of generality as any such 
offer will never be accepted. We do this so that we can (often) claim the uniqueness of optimal 
solutions. 

We first define what the optimal will be. To this end we first define 



Gjr >a (x) = 1 - \ a 



F(x) + xf{x) 
-, x r- V(x) 2 -F(y)f'(y) d - 

(yf(y) + F(y))*f(y)i dx 

As above, if G is a CDF, we will denote the corresponding density function by g. 

For the correct setting of a, Gj^ <a will be a (possibly infeasable) CDF of the local minimum. 
Gjr >a may be decreasing or negative and thus infeasible as a cumulative density function. We first 
construct a monotone version of Gj? a , then we make it positive. 

Next we define {Ik}- Because F is twice differentiable everywhere, 2/(x) 2 > f'(x)F(x) except 
on a countable number of intervals where 2/(x) 2 < f'{x)F{x). Let these intervals be {(ik,jk)}- 

For each (ik,jk) define 

Kk (x) = r fiy) fiy) dy 

i k = sup x < ik K k (x) < 0. 

to get the new interval (ik,jk)- Discard any intervals that are subsets of other intervals, and relabel 
the remaining intervals as I\ = [i\,ji),l2 = [^2,32), ■ ■ ■ where < i\ < j\ < i% < J2 ^ • • •• Let 
I = Ul k . 



Let 

GtAx) 



G?, a {x) x<£I 
Gr,a{Jk) x G J k 



xt, a. = max{0,inf / ffj a (x) < 1} 

x Jx 

G{x)r, a = max{G'jr iQ (x),0} 



Claim 4.1. G{x)jr a is a valid CDF: i.e. for x G [0, 00], G{x)jr a is non- decreasing, < G(x)j?, 
and lim^^oo G(x)jr !a = 1. 
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Proof, gjr a (x) = — — ^^ 3 x ' x , which is non-negative whenever 2f(x) 2 > Fix) fix) be- 

' n (x/(x)+F(x))2/(x)2 

cause F(x), /(x),x,n > 0. Thus, G\f )Q (x) is non-decreasing for x I because we have Gp^ a (x) = 
Gj^ >a (x) and 2/(x) 2 > F(x)f'(x). Also, for x £ 7 we have that <?j- iQ (x) = 0. This leaves only 
ifc and J& to check. By inspection (5j- iQ , is non-decreasing at j k for all /c (it is continuous and all 
surrounding derivatives are non- negative) . At any i k we have that Gj^ >a (ik) = Gj- )Q ,(j fc ), but that 
at all points to the left Gp^ a {i k — e) = Gp^ a (i k — e), which limits to Gp^ a {i k ) so we must show that 
Gj^ ta (ik) < Gj^ )a (jk)- However, this follows from the construction of I k because K k (i k ) = 0, but 
Kk(ik + e) > 0. Thus the derivative of K k at i k which is — f{ik)[{j{° g(z)dz)~ 2 — (J*-°° g(z)dz)~ 2 ] = 

-ftthdlO- - G(i k )y 2 - (1 - G(j k ))~ 2 ) is non-negative, and so Gjr, a (i k ) < Gjr^(j k ). 

Because G is non-decreasing, by construction G is also. Also, by construction < G(0). Because 
F{x)+lf{x) - x' for sufficiently large x we have 

I rv / f ( t) _ 

= G(x) < 1 

and so G{x) goes to 1 as x increases. □ 

Remark 4.1. We remark that Cost{F,Gjr^ a ,n) and V*(F,Gjr a ,n) are continuous in a and are 
monotonically increasing and monotonically decreasing in a, respectively. 

Theorem 4.2. Let F be twice differentiable for all positive x, and let n be fixed. The unique 
minimizer ofV£ n (G) subject to an expected cost constraint Costj? <n (G) < B, is of the form G r j-, a (x) 
where a is unique. Also, the unique minimizer of Costj^ )n {Q){G) subject to a worst-case expected 
variance constraint Vj n {G) < B, is of the form Gj^ )0l {x) where a is unique. 

Theorem 4.3. Let $(xi,X2) be continuous, convex, and monotone in both of its inputs; let F be 
twice differentiable for all positive x, andletn be fixed. The unique minimizer of ^(Cost{F,Q,n),V*{F,Q,n)) 
is of the form G , j-, a (x) where a is unique. Moreover, this is also guaranteed to be the global mini- 
mum. 

In this last theorem, the number of participants in the survey is no longer fixed, and we allow a 
cost to be associated with recruiting each participant independently of whether or not they accept. 

Theorem 4.4. Let F be twice differentiable for all positive x, and suppose there is a cost (3 > for 
recruiting each participant (i.e. independently of whether or not they accept the offer). If (3 > 0, 
then the unique minimizer ofV^(G,n) subject to an expected cost constraint Costjr(Q,n) < B, is 
of the form Gjr^ a {x) where a is unique. If (3 = 0, then there is a collection of minimizers of the 
form Gjr^ a (x) where a oc 1/n. 

Also, if f3 > 0, the unique minimizer of Costjr )n (Q) subject to a worst-case expected variance 
constraint Vj n {G) < B, is of the form Gj^ )Ct {x) where a is unique. If (3 = 0, then there is a 
collection of minimizers of the form Gj^ )0l {x) where \fa oc 1/n. 

Remark 4.2. The requirement that F be twice differentiable everywhere is for technical conve- 
nience. Our theorems can be generalized to the case where the PDF F is piece-wise twice differ- 
entiable by first approximating F with everywhere twice differentiable PDFs and then taking the 
limit. 
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We prove these theorems after developing the necessary machinery. 
Let 

r°° a r°° f(r) 

Mr,nA G ) = n / xF{x)g(x)dx + - / - n > dx (1) 

Jo n Jo l ~ <-* W 

H,, e „ Ax ) = nxF{x)+ ^j' iJ -m— dy (2) 

(3) 

For notational convenience, we will often omit the subscripts when their meaning is clear from 
the context (e.g. Mjr n \(G) and H{x)jr n ^\(G) become as M(G) and H(x), respectively). 

M(G) will be a function that we would like to minimize, 5Miq(G — G) is the derivative at G — G 
in the direction G — G, which we will want to show is always > at some local minimum. H (x) 
will turn out to be the derivative of M(G), a sort of point-wise version of 6M\q(G). 



Claim 4.5. 



X) 



6Mr, n ,x\G(G -G)= (g(x) - g(x))H(x)dx 
Jo 



Proof. This follows immediately from Claim 13.61 and Claim 13.71 □ 

Claim 4.6. Mjr^ n ^\(G) is strictly convex. 

Proof. By Claim [431 the first derivative of M\q is H(x) = nxF(x) + - J^ (\1q( x \\o. ■ Taking the 
second derivatives we get =^ J? hIq/ \\i dy, which is positive as long as we are above some point in 
the support of J- (because by Claim HTTI we know that G is between and 1). By our assumption, 
the only point less than every element in the support of J 7 that we consider is 0. Thus, the derivative 
at G in direction G: 

is positive unless G and the direction are the same except for at the point 0. However, this is never 
the case if the direction is non-trivial. And thus this is always strictly positive. □ 

Lemma 4.7. For all G, 8Mj7 n x\Q x (G — G) > 0. Moreover, G-p \ is the unique CDF G for 
which this holds. 

Proof. By Claim [4~5l we see that we must show that for all G, L (g(x) — g(x))H(x)dx > 0. Both 
g and g are distribution density functions, so they integrate to one and their difference integrates 
to 0. Necessarily off the support of G we have g{x) > g(x) = . Thus it is enough to show that 
H(x) is constant on some region that contains the support of G and that outside this region it is 
always greater than inside the region. 
We show that 

1. H is constant over the intervals [x,«i], \jk,ik+i] for 1 < i < m, and [j m ,oo); 

2. For every I k , H(i k ) = H(j k ) and for every x G I k , H(x) > H[j k ]; and 
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3. For i£ [0,x], H(x) > H{x). 
These three steps suffice to show that for all G, 5M :Fn uq (G — Gj- x ) > 0. 

_ _ n z 

Let x £ \j k , ik+i] for 1 < -i < ttt, then 

H(x) - H(j k ) = nxF{x) - nj k F(j k ) + - f / (j/) dy (4) 



n l j* 



{l-G T x{y)f 



= nxF(x) - nj k F(jk) + - f - / (y) , — dy (5) 

"4 (l-G^ A(y)) 2 

= (6) 

The last line follows because G-p x (x) = 1 — J ^ f(J)+x fix) so 

- I Tr^^T^ dy = - (irvfW + F (2/)) = nxF W ~ n ~JkFQ k )- 

nj' jk (1-Gjr x(y)) 2 »V A /j fc 

At the single point ik+i, it might not be that G-p x (y) = G-p x (y), but because this is a single 

point, and f(x) is differ entiable, this will not affect the calculation. 
The same argument works for x £ [x,i\], and x £ [j m ,oo) 
To show 2) observe that: 



nj x (l-G(y)f 



H(j k ) ~ H{x) = nj k F(jk) ~ nxF(x) + - / - J %' dy (7) 



= nl k FG k ) - nxF(x) + * T /fa) dy + * ['" — f -M M dy 

(8) 

_ A F Jk f(y) f(y) 

~nj x {l-G(y)Y~ {^g(z)dzY V (9) 

However, by the definition of I k , for i k < x < j k this is less that or equal to with equality 
when x = i k . 

Finally, we show that for x £ [0, x], H(x) < H(x). As in the previous setup we see that: 

H(£) _ H(x) = * f jk IM M dv m 

But this is negative because for x < x we have that J^° g(z)dz > 1, but G(y) = 0. 
The uniqueness follows directly from the fact that we found a local minimum and by Claim 14.61 
the function is strictly convex over our space which is also convex. □ 
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of Theorem \4-%\ '■ Let F be twice differentiable for all positive x, and let n be fixed. We minimize 
V*(G) subject to a expected cost constraint B. From Lagrange Multipliers (see section 12, 2D we 
have 



L[G, A] = V> in (G) + A (Cost^ n (g) - B 

= I r r f(y) 



Thus we have that: 



I I nt \ d y dx + An ( I (xF{x)g{x)dx - B 

Jo Jo 1" G(y) \Jo 

dL[G,X] 



dG 

dL[G, A] 



= 5M Tnj _(G) >0 (11) 

nf™xF(x)g(x)dx-B =0 (12) 



where M is define in Equation [TJ By Equation [12] we see that L nxF{x)g-p n i{x)dx = B/n 
and by Remark 14. II this uniquely determines A. From Lemma 14.71 we see that Equation 1111 implies 
that G = Gjr i . 

The proof is analogous when we are minimizing the cost with a bound on the variance. □ 

of Theorem \4-3\ : 

<J$| G (G -G) = ^- 5Cost(G -G) + ^- 5V£(G - G) = 5M F ^ X{G (G - G) 

OXx\Cost{G) OXx\v*{G) 

where n = §^-, r ,„, and A = ^- . .n. So for any fixed Gj- )Q to be optimal we must have 

o~M Fn x\G{G ~ Gt,o) > for all G. However, by Lemma I4T71 we see that if this is the case, then G 
must be of the form G F}a (x) for some 

A dx 2 \V*(G Tta ) 

a - 



n 2 9 ® 

dxi\Cost{G TiCl ) 

The RHS, a, is strictly monotonically increasing in a and goes from to infinity as a does, 
where as the LHS is monotonically decreasing with respect to a (because V*{G F ^ a ) decreases with a 
and Cost* (G^ a ) increases with a, and $ is convex), and is always positive because <l> is monotone. 
Because all the functions are continuous, there exists a unique solution for a. Because <£, Cost and 
V* are all convex the composition is, and thus this is a global maximum. □ 

Now we generalize the results to the setting where the number participants is not fixed a priori, 
but there may be some cost (5 associated with each additional person recruited. 



of Theorem \4-4\ '■ 

Thus the cost of the mechanism becomes 



Cost(.F, Q,n,P)=n(p+ I xF{ 



x)g{x)dx 

and the variance remains the same. We will deal with the setting where we would like to minimize 
the variance subject to some cost constraint. This give us the following problem: 
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.,■ ■ • 1 f°° fix) 
Minimize — ' 



n J 1 — G(x) 
Subject to: n ( (3 + / c • g(c) ■ F(c)dc) = B 
Using a Lagrange multiplier, we arrive at the equation: 

L[G, A] = V^ n (G) + A (Cost^ )n (a) - B) 
1 'f°° f(x 



n J 1 - G(x] 
Thus we have that: 

dL[G,n,X] 



dx + A ( n ( (3 + / (xF(x)g{x)dx) - B j 



= ^^,l(G) >0 (13) 

^ [ g; n,A] =^^ T %r ) dx + \(P + ^xF{x) 9 {x)dx) =0 (14) 

<:ALKt '- "• A| =n(/3 + / °° xF(^) 5 (x)(ix) - 5 =0 (15) 



dX 

where M is define in Equation [TJ 

From Lemma 14.71 we see that Equation [13] implies that G = G-p i . If we let a = -r— ^ , then 
Equation 1141 then implies that 

V^(V> x (G T , a )) = -1= (Cost^i (Gjr lQ ) + /3). (16) 

However, 
v^CV>,i(Gjr ia )) = V^ 



^ /to dx + ( /"'" /(*) _ /(*) ^ 

o f™ 9T, a (y) d y \Jo IT 9T, a (y) d y ITfcAvWv 



o $r 9T,i{y)dy * vio XT 9T, a {y) d y IT 9rAy) d y 

And this increases in a because, by the definition of xj-q,, the integrand in the second term is 
positive. Similarly, 






xF(x)gjr >a (x)dx - f / xF{x)gjr >a (x)dx J + /3 

= / xF(x)gjr 1 (x)dx- / xi ?, (a)^jri(x) + — F =rfx 

Jo Jo V a 

And this decreases to as a increases. Moreover, if (3 > then the RHS monotonically decreases 
from +oo to 0, and so there exists a unique a that satisfies Equation [16l Having derived a value 
for a, Equation [T5l uniquely determines n. 
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In the other case, /3 = 0, and we can solve with different values of n, as in Theorem 14.21 and 
directly compare the values of the objective function. As we have seen, the optimal G is of the 
form Gjr a where a = -4r • From Equation [12] (or Equation [T4"|) and using a derivation similar to 
that above, we see that 

y/\ = B/[ / xF(x)gjri(x)dx - \ xF(x)gr t i(x) + —= dx ) , 
\Jo Jo V« / 

and the objective function becomes 

1 r m to_rt(r_m dx+ ^(r" /feL -fML dx 



/)./„ l-G T>a (x) ' \J f™ gr,i(y)dy ' \J f™ 9T )a {v) d V IT 9T,a(y) d V 

r°° f( x ) dx + fa ( r 2jr ' Q i-oo /^ /(g) 

JT 9T,i(y)dy v \J0 JT 9T, a (y) d y JT ' 9F,a(y)dy 



B- 



J °° xF(x)gjr A (x)dx - J^' a xF(x)gjr^(x) + -^dx 



This is minimized if xq = because the extra term in the numerator is positive and the extra 
term in the denominator is negative. In the degenerate case where xj^ tOL is always greater than 0, a 
solution may not exist. This happens if and only if there is mass on T arbitrarily close to 0. The 
reason for this is that the surveyor will want an unbounded number of samples close to 0, because 
no matter how many samples he has, their price to variance trade off is cheap because their price 
is arbitrarily small. Otherwise, after n is above some threshold a will be sufficiently small so that 
xo = 0. At this point, the objective function will remain the same regardless of n, and there exists 
a inverse relationship between ^fa and n because A is fixed and a = ■%. 

□ 

5 Discussion and Open Problems 

In this paper, we have initiated the study of truthful, Bayesian-optimal survey design for generating 
unbiased estimators of a population statistic. Of course, there are many interesting directions left 
open for future work. For example: 

1. In this paper, we have focused on mechanisms which promise unbiased estimators, and have 
focused on optimizing properties of these mechanisms (i.e. variance, cost, etc.). However, 
if the surveyor is in the end interested in accuracy, he may be willing to compute a biased 
estimator if doing so can significantly reduce variance without increasing cost (error, in the 
end, is a function both of the bias of the estimator, and of its variance). What do optimal 
mechanisms look like in the regime of minimizing error, when we cannot necessarily restrict 
our attention to the Horvitz-Thompson estimator? 

2. We have assumed a known prior over costs, but have otherwise made worst-case assumptions 
on what the distributions on surveyed types looks like. Is it possible to do substantially 
better in some cases if we make further (natural) assumptions on the distribution from which 
player types are drawn? One natural assumption might be that E[q , (x)|c(x)] is monotonically 
increasing in c{x). For example, we would expect this to be the case if q(x) = 1 represents 
some stigmatized property, and so such individuals might have higher cost for participating 
in such a survey. 
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3. What if costs are indeed drawn from some prior distribution, but this distribution is un- 
known to the mechanism. Can a mechanism which must now learn this distribution over 
costs compete with the optimal mechanisms derived here, with prior knowledge over the true 
distribution? 
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